Method for providing telepresence using avatars, and system and computer-readable recording medium using the same

ABSTRACT

A method for providing telepresence by employing avatars is provided. The method includes steps of: (a) a corresponding location searching part determining a location in a first space where an avatar Y′ corresponding to a human Y in a second space will be placed, if a change of a location of the human Y in the second space is detected from an initial state, by referring to (i) information on the first space and the second space and (ii) information on locations of the humans X and Y, and the avatar X′ in the first and the second spaces; and (b) an avatar motion creating part creating a motion of the avatar Y′ by referring to information on the determined location where the avatar Y′ will be placed.

CROSS REFERENCE TO RELATED APPLICATION

This application claims priority to and incorporates herein by referenceall disclosure in Korean Patent Application No. 10-2015-0003706 filedJan. 9, 2015.

FIELD OF THE INVENTION

The present invention relates to a method for providing telepresence byusing avatars and a system and a computer-readable recording media usingthe same; and more particularly, to a technology for changing a motionadaptively by recognizing information on space and consideringinformation on a location, an orientation, etc. of an object in thespace to implement an effect of enabling a first person in a first spaceand a second person in a second space to gather in the first spacethrough an avatar of the second person and/or gather in the second spacethrough an avatar of the first person.

BACKGROUND OF THE INVENTION

A telepresence system by using avatars may be applied to a technologyfor allowing participants at distant places to feel as if they were atone place.

Accordingly, the technology allows people who are far away from eachother to feel as if they were together at his or her present space byusing avatars.

Therefore, technologies relating to virtual video calls by using avatarshave been disclosed in Korean Laid-Open Publication No. 2005-7017989,U.S. Pat. No. 5,736,982 and United States Laid-Open Publication No.2008-0082311.

According to the Korean Laid-Open Publication No. 2005-7017989, atechnology for allowing a user to control a pose of his or her avatareasily by using a keyboard, mouse, facial expression, motion, etc. isdisclosed. Besides, according to the U.S. Pat. No. 5,736,982, atechnology for identifying an intention of a user who accesses a virtualworld to talk with another avatar (user) by using information on theuser's gaze and his or her avatar's location, and then connecting aspeech channel between the two users is disclosed. According to the U.S.Laid-Open Publication No. 2008-0082311, a technology for allowing a userto invite a person at a remote place in a present virtual space and playa computer game or watch a TV together with the person is disclosed.

Accordingly, the conventional technologies can control avatars andmutually connecting them to each other but they have a problem in thatthey do not control their motions adaptively in an environment where theusers can co-exist at their own spaces and take their poses naturally.

Therefore, the inventor came to develop a technology for controlling amotion of a user's avatar adaptively even in another person's virtualspace.

SUMMARY OF THE INVENTION

It is an object of the present invention to solve all the aforementionedproblems.

It is another object of the present invention to allow a human in apresent space and an avatar in the present space corresponding toanother human in a remote place to naturally communicate with each otheradaptively.

It is still another object of the present invention to naturally outputan avatar's motion corresponding to a human's motion by reflectingcharacteristics of objects in the present space and the remote place.

In accordance with one aspect of the present invention, there isprovided a telepresence system by using avatars, including: acorresponding location searching part for determining a location in afirst space as a present space where an avatar Y′ corresponding to ahuman Y in a second space as a remote place will be placed, if a changeof a location of the human Y in the second space is detected from aninitial state in which an initial location of a human X in the firstspace, that of the human Y in the second space, that of the avatar Y′ inthe first space corresponding to the human Y, and that of an avatar X′in the second space corresponding to the human X are determined, byreferring to (i) information on the first space and the second space and(ii) information on locations of the humans X and Y, and the avatar X′in the first and the second spaces; and an avatar motion creating partfor creating a motion of the avatar Y′ by referring to information onthe determined location where the avatar Y′ will be placed.

In accordance with another aspect of the present invention, there isprovided a method for providing telepresence by employing avatars,including steps of: (a) a corresponding location searching partdetermining a location in a first space as a present space where anavatar Y′ corresponding to a human Y in a second space as a remote placewill be placed, if a change of a location of the human Y in the secondspace is detected from an initial state in which an initial location ofa human X in the first space, that of the human Y in the second space,that of the avatar Y′ in the first space corresponding to the human Y,and that of an avatar X′ in the second space corresponding to the humanX are determined, by referring to (i) information on the first space andthe second space and (ii) information on locations of the humans X andY, and the avatar X′ in the first and the second spaces; and (b) anavatar motion creating part creating a motion of the avatar Y′ byreferring to information on the determined location where the avatar Y′will be placed.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other objects and features of the present invention willbecome apparent from the following description of preferred embodimentsgiven in conjunction with the accompanying drawings, in which:

FIG. 1 is a concept diagram of a telepresence system in accordance withone example embodiment of the present invention.

FIG. 2 is a whole configuration of the telepresence system in accordancewith one example embodiment of the present invention.

FIG. 3 represents a state of dividing spaces into patches to searchcorresponding locations under the telepresence system in accordance withthe present invention.

FIG. 4 illustrates a state of information assigned to individualpatches.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

In the following detailed description, reference is made to theaccompanying drawings that show, by way of illustration, specificembodiments in which the invention may be practiced. These embodimentsare described in sufficient detail to enable those skilled in the art topractice the invention. It is to be understood that the variousembodiments of the present invention, although different, are notnecessarily mutually exclusive. For example, a particular feature,structure, or characteristic described herein in connection with oneembodiment may be implemented within other embodiments without departingfrom the spirit and scope of the present invention. In addition, it isto be understood that the position or arrangement of individual elementswithin each disclosed embodiment may be modified without departing fromthe spirit and scope of the present invention. The following detaileddescription is, therefore, not to be taken in a limiting sense, and thescope of the present invention is defined only by the appended claims,appropriately interpreted, along with the full range of equivalents towhich the claims are entitled. In the drawings, like numerals refer tothe same or similar functionality throughout the several views.

To allow those skilled in the art to the present invention to carry outeasily, the example embodiments of the present invention by referring toattached diagrams will be explained in detail as follows:

FIG. 1 is a concept diagram under a telepresence system by using avatarsin accordance with one example embodiment of the present invention.

By referring to the telepresence system of FIG. 1, an initial statewhere there are a human X in a first space as a present space, a human Yin a second space as a remote place, an avatar Y′ (corresponding to thehuman Y) in the first space, and an avatar X′ (corresponding to thehuman X) in the second space is represented. A method for determiningthe initial state will be explained later.

Under such a situation, if a change of a location of the human Y in thesecond space is detected, a corresponding location searching part 240 tobe explained below may determine a location where the avatar Y′ will beplaced in the first space by referring to (i) information on the firstspace and the second space and (ii) information on locations of thehumans X and Y and the avatar X′ in the first and the second spaces.Herein, the corresponding location searching part 240 may also determinethe location where the avatar Y′ will be placed in the first space byadditionally referring to (iii) information on gazes of the humans X andY in the first and the second spaces. Besides, an avatar motion creatingpart 250 to be explained later may create a motion of the avatar Y′ byreferring to information on the location where the avatar Y′ will beplaced.

Hereinabove, only the avatar Y′ was mentioned in detail, but it isnatural that this could be applied equally even to the avatar X′.Further, the first space was mentioned as a present space and the secondspace was mentioned as a remote space but the first space could be theremote place and the second space could be the present space. However,for convenience, only one space will be explained.

For reference, the avatars X′ and Y′ may be virtual avatars visualizedthrough head mounted displays (HMDs) worn by the humans X and Y andother augmented reality (AR) displays or physical avatars such asrobots. At the time, appearances of the avatars could resemble humans atthe remote places or be expressed as other figures such as animals orcharacters.

In addition, the present invention assumes a situation under which notonly the structures of the present space and the remote place but alsoshapes, quantity, types, and layouts of objects such as furnitureexisting in the respective spaces could be different.

For example, if the human Y who stood in front of a door of the remoteplace approaches, and sits on, a chair at a different location in theremote place, the human Y at the remote place is allowed to be adaptedto an environment of the present space by making the avatar Y′ used torepresent the human Y take an action of standing in front of a door ofthe present space and then approaching a chair at a proper location inthe present space. This will be explained later in more detail.

Furthermore, as explained above, as it was assumed that the structuresof the present space and the remote place and the shapes, thequantities, the types, and the layouts of the objects in the presentspace and the remote place could be different, there could be a varietyof routes of reaching the destination for the avatar Y′ in the presentspace according to the change of the motion of the human Y in the remoteplace.

For one example, even though the human Y at the remote place walked in astraight line to move to the chair from the front of the door, theavatar Y′ at the present space which has a structurally differentenvironment from the remote place may take an action of turning aroundor getting over an obstacle to go to the chair from the front of thedoor. This will be also explained later in detail.

A whole configuration of the telepresence system by using avatars in thepresent invention based on the aforementioned concepts is illustrated inFIG. 2.

As shown in FIG. 2, a telepresence system 200 by using avatars in thepresent invention includes a space recognition part 210, a human motionrecognition part 220, and a human gaze recognition part 230 foracquiring a variety of pieces of information to recognize actualconditions of a human and an environment, a corresponding locationsearching part 240 for searching a corresponding location, i.e., anoptimal location where an avatar will be placed in the present space, anavatar motion creating part 250 for receiving a value from thecorresponding location searching part 240 to map a motion of the human Yin the remote place to the avatar Y′ in the present space, a userinterface providing part 260 for allowing a user to determine whether anestimated destination is appropriate or not, a communication part 270,and a control part 280. For reference, the telepresence system 200 inaccordance with the present invention necessarily include thecorresponding location searching part 240, the avatar motion creatingpart 250, the communication part 270, and the control part 280.Meanwhile, the space recognition part 210, the human motion recognitionpart 220, the human gaze recognition part 230, and the user interfaceproviding part 260, if necessary, may be included in the telepresencesystem 200 in accordance with the present invention or separately inanother system.

The space recognition part 210 measures data by means of various typesof sensors, including an image sensor, a depth sensor, a laser sensor,etc. and acquires information on types of objects such as furniture(e.g., a chair, a table, a bed, etc.) and space information on locationsand orientations of the objects (i.e., directions on which the objectsare placed, etc.).

At the time, the space information may include information on types,locations, and orientations of the objects granted to the patches, whichwill be explained later.

To acquire information on a location of a human, the human motionrecognition part 220, moreover, could measure data with respect to ahuman motion by means of a wearable sensor such as an image sensor, adepth sensor, or an inertial measurement unit (IMU) and acquireinformation on a location and a motion of the human as a function oftime through the measured sensor data.

Besides, the human gaze recognition part 230 acquires information on anobject gazed by a human and a gaze point through a variety of sensordata with which gaze measurements are available.

The corresponding location searching part 240 calculates an appropriatelocation where the avatar Y′ in the present space corresponding to thatof the human Y in the remote place will be placed or a changing locationof the avatar Y′ in the present space, if the human Y in the remoteplace is changed his or her location within the remote place, based onthe acquired information on the space, the location, the motion, and thegaze.

In other words, it calculates an optimal corresponding location in thepresent space where the avatar Y′ will be placed according to themovement of the human Y by receiving 1) information on the remote placeand the present space, 2) information on locations of humans in theremote and the present spaces and/or 3) gaze information and thenestimating a destination where the human in the remote place intends toapproach.

At the time, the corresponding location searching part 240 may dividespaces and surfaces of objects into virtual patches, in which case theestimation of the destination and the calculation of the optimalcorresponding location may be performed in a unit of patch.

The patches are created by dividing the spaces in a unit of plane.Herein, a curved surface may be divided into multiple planes. Bydetermining a minimum value (e.g., 0.1 1 m²) and a maximum value (e.g.,1 m²) of an area of each patch to allow the patches to express alocation of a human properly, a plane not more than the minimum valuemay be controlled not be included in the patch and that not less thanthe maximum value may be controlled to be divided into multiple patches.

The corresponding location searching part 240 gives and stores an ID ofan object and geometrical information (including sizes, heights, anddirections of normal vectors) with respect to each patch according tospace information. Besides, each patch has information on a location anda principle direction in a space according to the space information.This will be stated by referring to FIG. 3.

By referring to FIG. 3, if s_(A, i) is a patch i corresponding to aspace A, s_(A, p) corresponds with a patch P which corresponds to thespace A and s_(A, q) corresponds with a patch Q which corresponds to thespace A.

On the assumption that a location of s_(A, i) is p_(A, i) and aprincipal direction thereof is d_(A, i), if the corresponding patch hasa principal direction such as a chair, the principal direction d_(A, i)is designated as a direction taken principally by a human and if thecorresponding patch has no principal direction such as a floor, theprincipal direction d_(A, i) is designated as “undefined.”

As shown in FIG. 3, a principal direction d_(A, q) of a patch s_(A, q)corresponding to a chair is represented as an arrow head and a principaldirection d_(A, p) of a patch s_(A, p) corresponding to a floor isundefined, which means that there is no corresponding arrow head.

Meanwhile, the corresponding location searching part 240 calculates anoptimal corresponding location, i.e., a patch and a direction of thepatch where an avatar should be placed at an initial state when theavatar is initially operated as shown below. This will be explained inmore detail by referring to FIG. 4.

By referring to FIG. 4, on the assumption that a location and anorientation of the human X in the present space A are p_(A, i) andθ_(A, i) and those of the human Y in the remote place are p_(B, j) andθ_(B, j), an optimal patch s_(A, j′) and an optimal orientationθ_(A, j′) of the avatar Y′ (corresponding to the human Y in the remoteplace) to be placed in the present space are determined such that adegree of intrinsic similarity and a degree of spatial proximity reachmaximum values.

The degree of intrinsic similarity represents a degree of similaritywith respect to (i) information on types of objects (e.g., a chair, atable) in the patches and (ii) geometrical characteristics (e.g., normalvectors, patch sizes, heights, and aspect ratios) of the patches.

The degree of spatial proximity may be determined as a function whichincreases when p_(A, i) and p_(A, j′) are closer to a certain distance α(determined as a proper distance that people can talk with each other,e.g., 1 m) and θ_(A, j′i) (i.e., a difference between θ_(A, j′) andθ_(A, i)) is closer to 180 degrees to make a human and an avatar faceeach other. For example, information on the degree of spatial proximitymay be determined as a function shown below.(p _(A,j′) ,p _(A,i),θ_(A,j′i))=exp(−λ₁(∥p _(A,j′) p_(A,i)∥−α)−λ₂|θ_(A,j′i)−180|)

where λ_(i) (≧0) is a weighting factor which controls the influence ofeach term.

The initial optimal corresponding location may be determined as alocation with high degrees of intrinsic similarity and spatialproximity. For example, the location may be determined as a locationwhere the following fitness function is maximized:max{(1−w ₂)·intrinsic_similarity(s _(A,j′) ,s _(B,j))+w₂·spatial_proximity(p _(A,j′) ,p _(A,i),θ_(A,j′i))}

where w₂ (≧0) is a weighting factor which controls the influence of theintrinsic similarity and the spatial proximity.

Similarly, a location p_(B, i′) and orientation θ_(B, i′) of the avatarX′ in the remote place could be obtained by the aforementioned method.

Besides, it could be also assumed that, if the present and the remoteplaces are divided into multiple patches, the corresponding locationsearching part 240 allows the avatar Y′ in the present space to have adirection of 180 degrees (or near 180 degrees) from a direction of thehuman X and have a location within a predetermined distance from thehuman X to thereby be placed in a patch with a high degree ofappropriateness for a human to be located in the present space, andallows the avatar X′ in the remote place to have a direction of 180degrees (or near 180 degrees) from the a direction of the human Y andhave a location within a preset distance from the human Y to thereby beplaced in a patch with a high degree of appropriateness for a human tobe located in the remote place.

By the afore-stated method, the initial states of the locations of thehuman X and the avatar Y′ in the present space and the human Y and theavatar X′ in the remote place would be determined. As shortly mentionedabove, the states in FIG. 1 are examples of the initial states of thehuman X and the avatar Y′ in the present space and the human Y and theavatar X′ in the remote place.

Meanwhile, it is assumed that the human Y in the remote place changeshis or her location from the initial state of the location of the humanY. In the case, the corresponding location searching part 240 maycalculate an appropriate location where the avatar Y′ in the presentspace will be placed when it detects the changed location of the human Yin the remote place.

First of all, to calculate the appropriate location where the avatar Y′in the present space will be placed, a degree of similarity betweenpatches of the present space and those of the remote place should becalculated.

In other words, the degree of similarity between patches of the presentspace and those of the remote place is obtained by calculating a degreeof intrinsic similarity and a degree of spatial similarity.

The degree of spatial similarity is acquired by referring to (i)information on a relative location and a relative direction between thehuman X and the avatar Y′ in the first space and (ii) information on arelative location and a relative direction between the human Y and theavatar X′ in the second space.

First of all, on the assumption that a human and an avatar are placed intwo patches s_(A, i) and s_(A, j) in the space A in FIG. 4 withspecified orientations θ_(A, i) and θ_(A, j′) respectively (arrow headsin FIG. 4), θ_(A, i) may be equal to or different from d_(A, i), or maybe designated as undefined as the case may be.

At the time, a coordinate system may be set based on p_(A, i) andθ_(A, i) and a relative location of s_(A, j′) in reference to the setcoordinate system is defined as p_(A, j′i) and a difference between theorientations θ_(A, i) and θ_(A, j′) in reference to the set coordinatesystem is defined as θ_(A, j′i) (=θ_(A, j′)−θ_(A, i)). If either ofθ_(A, i) and θ_(A, j′) is undefined, θ_(A, j′i) may be determined asundefined. s_(B, ji′) and θ_(B, ji′) may be determined in the space B inthe similar manner.

Accordingly, a degree of spatial similarity between s_(A, j′) ands_(B, j) with respect to reference patches s_(A, i) and s_(B, i′) isdetermined as a function which increases as the differences betweenp_(A, j′i) and p_(B, ji′) and between θ_(A, j′i) and θ_(B, ji′) becomesmaller. For example, the degree of spatial similarity may be calculatedas follows:spatial_similarity=exp(−λ₁ ∥p _(A,j′i) −p_(B,ji′)∥−λ₂|θ_(A,j′i)−θ_(B,ji′)|)

where λ_(i) (≧0) is a weighting factor which controls the influence ofeach term.

At the time, if both θ_(A, j′i) and θ_(B, ji′) are undefined,θ_(A, j′i)−θ_(B, ji′) may be determined as 0 and if either of them isundefined, a sufficiently large value may be given to|θ_(A, j′i)−θ_(B, ji′)|.

Next, a method of the corresponding location searching part 240calculating an appropriate location where the avatar Y′ in the presentspace will be placed when a change of a location of the human Y in theremote place is detected will be explained in detail.

If detecting a change of location of the human Y in the remote place,the corresponding location searching part 240 estimates a location whichthe human Y (moving) in the remote place intends to reach, i.e., a patch(s_(B, j)) as a destination.

To estimate the patch (s_(B, j)) as the destination, the correspondinglocation searching part 240 may refer to information on probabilities ofthe human Y reaching the respective patches and then determine a patchwith the highest probability as the patch (s_(B, j)), i.e., thedestination. The probabilities P_(tp) (s_(B, k)) (k=1, 2, . . . ) of thehuman Y reaching the respective patches as the destination, i.e.,s_(B, k) (k=1, 2, . . . ), may be obtained as shown below. Herein, theprobabilities P_(tp) (s_(B, k)) (k=1, 2, . . . ) of the human Y reachingthe respective patches as the destination means probabilities of therespective patches being determined as the destination which the human Yintends to reach.P _(tp)(s _(B,k))=gaze_term*affordance_prior_term*humanal_prior_term

where the gaze term is a function which increases as a cumulative value(an integral value) of probabilities of gazing at a patch s_(B, k)during a time section (e.g., one second) from the past to the presentincreases.

Furthermore, the affordance prior term is a function for evaluatingdegrees of appropriateness of a human to be located by referring totypes of objects to which the patches correspond and geometricalcharacteristics of the patches (e.g., heights, normal vectordirections). For instance, the types of objects such as a chair, or afloor where the human is easy to be located have relatively highervalues than those of objects such as a table or a desk. Further, if anormal direction of an object is more distant from the oppositedirection of gravity, a relatively lower value may be assigned.Additionally, the affordance prior term has a smaller value when aheight of an object is higher than a reference height difficult for thehuman to be located (e.g., 1.5 meters from the floor) or when an emptyspace on a patch is lower than a certain reference height (e.g., 1meter).

In addition, the personal prior term is a function which increases as afrequency of a patch s_(B, k) being the destination actually reached bythe human Y increases.

Accordingly, s_(B, j) could be selected as a patch with the largestP_(tp) among the patches in the space B (k=1, 2, . . . ), and it couldbe explained under the following formula:s _(B,j)=arg max_(k) P _(tp)(s _(B,k))

Meanwhile, the corresponding location searching part 240 may deliverinformation on s_(B, j) as a patch with the largest P_(tp) as shownabove through the user interface providing part 260 of the human Ymoving in the remote place. At the time, the user interface providingpart 260 may include an AR display such as an HMD and the human Y maygive feedback about whether the estimation result, i.e., the informationon s_(B, j), is appropriate or not by using the user interface providedby the user interface providing part 260. For example, if s_(B, j) isnot a destination the human Y intends to move and reach, the human Y maygive a negative feedback through the user interface providing part 260in a meaning of wrong estimation. Then, the corresponding locationsearching part 240 may estimate another destination candidate (until thehuman Y gives a positive feedback) through an iterative process, i.e.,through a course of repeatedly delivering information on a patch with anext highest probability to the user interface providing part 260 untila positive feedback is acquired from the human Y.

Next, on assumption that the estimated destination patch for the human Ymoving in the remote place is designated as s_(B, j), the location andthe orientation of the avatar X′ in the remote place are p_(B, i′) andθ_(B, i′), and the location and the orientation of the human X in thepresent space are p_(A, i) and θ_(A, i), an optimal patch s_(A, j′) andan optimal orientation θ_(A, j′) of the avatar Y′ (i.e., the avatar forthe human Y in the remote place) can be calculated to thereby acquire anoptimal corresponding location for the avatar Y′.

θ_(B, j) is determed as d_(B, j) (a principal direction of s_(B, j)) andif d_(B, j) is undefined, even θ_(B, j) is designated as undefined.

s_(A, j′) and θ_(A, j′) are selected such that the degree of intrinsicsimilarity and the degree of spatial similarity as specified abovebecome maximums.

For example, s_(A, j′) and θ_(A, j′) are selected such that thefollowing fitness function becomes maximum:max{(1−w ₁)·intrinsic_similarity(s _(A,j′) ,s _(B,j))+w₁·spatial_similarity(p _(A,j′i) ,p _(B,ji′),θ_(A,j′i),θ_(B,ji′))}

w₁ (≧0) as a scalar is a weighting factor which controls degrees ofimportance with respect to the intrinsic similarity and the spatialsimilarity.

For reference, by the aforementioned optimization course, if θ_(B, j) isundefined, a patch whose θ_(A, j′) is undefined has a higher probabilityto be selected as s_(A, j′) and if θ_(b, j) is defined, a patch whoseθ_(A, j′) is defined has a higher probability to be selected ass_(A, j′).

Meanwhile, the avatar motion creating part 250 may create a motion ofthe avatar Y′ by referring to information on the estimated destinationwhere the avatar Y′ will arrive. To do this, the avatar motion creatingpart 250 may be operated all the time to thereby create the movement ofthe avatar in the present space according to the movement of the humanin the remote place.

The avatar motion creating part 250 outputs information on a shape of anavatar which takes a pose after receiving and analyzing at least someof 1) an output value of the corresponding location searching part 240,2) information on a human motion, a gaze and a space measured byrecognition sensors (i.e., a human motion recognition part, a human gazerecognition part, and a space recognition part) in the remote place, 3)information on a human motion, a gaze and a space measured byrecognition sensors in the present space, and 4) information on a basicshape (including body size, appearance, etc.) of the avatar.

When the corresponding location searching part 240 determines thedestination where the avatar will be placed, the avatar motion creatingpart 250 searches an optimal route to make the avatar avoidinterpenetration (i.e., collision) and creates a moving motion of theavatar.

There may be a lot of appropriate algorithms for searching a route andcreating a moving motion.

At the time, the motion of the avatar may be created by referring to atype of the motion (e.g., sitting, standing, lying, or gesturing) of thehuman in the remote place. Due to a difference in shape between thehuman and the avatar and a difference in environment between the remoteplace and the present space, if a pose (e.g., angles of respectivejoints) of the human in the remote place is applied to the avatar in thepresent space as it is, the interpenetration with the environment (e.g.,furniture, etc.) may occur. Therefore, the orientation of the avatar isdetermined to avoid the interpenetration with the environment while theorientation of the human in the remote place is kept to the maximum.

For example, a revised pose of the avatar may be determined by solvingan optimization problem as shown below.min{(1−w ₃)·pose_difference+w ₃·penetration_penalty}

where pose_difference means a value which becomes larger when thedifference in poses between the human and the avatar becomes larger andpenetration_penalty means a value which becomes larger when a portion ofthe avatar penetrating into the environment becomes more. Of course,explanation on the algorithm for calculating a degree of penetrationbetween two objects is omitted because there are many such algorithms.

In accordance with the present invention, if a corresponding locationsearching part displays a new destination where an avatar will beplaced, the present invention has an effect of the avatar searching anoptimal route to avoid interpenetration with the environment and thencreating its moving motion.

The present invention also has an effect of creating a motion of anavatar in the present space corresponding to a type of a motion of ahuman in a remote place.

The embodiments of the present invention as explained above can beimplemented in a form of executable program command through a variety ofcomputer means recordable to computer readable media. The computerreadable media may include solely or in combination, program commands,data files, and data structures. The program commands recorded to themedia may be components specially designed for the present invention ormay be usable to a skilled human in a field of computer software.Computer readable record media include magnetic media such as hard disk,floppy disk, and magnetic tape, optical media such as CD-ROM and DVD,magneto-optical media such as floptical disk and hardware devices suchas ROM, RAM, and flash memory specially designed to store and carry outprograms. Program commands include not only a machine language code madeby a complier but also a high level code that can be used by aninterpreter etc., which is executed by a computer. The aforementionedhardware device can work as more than a software module to perform theaction of the present invention and they can do the same in the oppositecase.

As seen above, the present invention has been explained by specificmatters such as detailed components, limited embodiments, and drawings.While the invention has been shown and described with respect to thepreferred embodiments, it, however, will be understood by those skilledin the art that various changes and modification may be made withoutdeparting from the spirit and scope of the invention as defined in thefollowing claims.

Accordingly, the thought of the present invention must not be confinedto the explained embodiments, and the following patent claims as well aseverything including variations equal or equivalent to the patent claimspertain to the category of the thought of the present invention.

What is claimed is:
 1. A telepresence system by using avatars,comprising: a space recognition part including at least one sensor, saidspace recognition part acquiring information regarding at least a firstspace, which is a present space, and a second space, which is a remotespace; a human motion recognition part including at least one sensor,said human motion recognition part acquiring information on locations ofa human X in the first space and a human Y in the second space; (A) acorresponding location searching part for determining a location in thefirst space where an avatar Y′, by referring to (i) information on thefirst space and the second space, (ii) information on locations of thehumans X and Y and an avatar X′ in the second space, and (iii)information on gazes of the humans X and Y in the first and secondspaces; wherein the avatar Y′ corresponds to the human Y, and the avatarX′ corresponds to the human X, wherein the corresponding locationsearching part estimates the destination in a unit of patch by dividingat least part of the first and the second spaces into patches, anddetermines the location where the avatar Y′ will be placed, wherein thecorresponding location searching part determines the location in thefirst space where the avatar Y′ will be placed by selecting a locationthat has the highest degree acquired by referring to (i) information ona degree of intrinsic similarity calculated by using at least some ofinformation on types of objects corresponding to the patches included inthe first and the second spaces and at least some of information ongeometrical characteristics of the patches; and (ii) information on adegree of spatial similarity calculated by using information on arelative location and a relative direction between the human X and theavatar Y′ in the first space and those between the human Y and theavatar X′ in the second space, wherein the corresponding locationsearching part determines a destination in the second space where thehuman Y intends to reach from the initial state by referring to at leastpart of (1) information on a function which increases as cumulativevalues of respective probabilities of gazing at the respective patchesduring a certain time section increase, (2) information on a functionfor evaluating degrees of appropriateness of locations for the human Yto be located by considering types of objects to which the respectivepatches correspond and geometrical characteristics of the respectivepatches, and (3) information on a function which increases asfrequencies of the respective patches being the destination actuallyreached by the human Y become higher; and (B) an avatar motion creatingpart for creating a motion of the avatar Y′ by referring to informationon the determined location where the avatar Y′ will be placed; whereinthe avatar motion creating part creates a motion of the avatar Y′ bysearching a route to avoid interpenetration in the first space, andwherein the avatar motion creating part creates the motion of the avatarY′ by referring to (i) a degree of difference between a pose of thehuman Y and that of the avatar Y′ and (ii) a degree of penetration ofthe avatar Y′ into an environment of the first space.
 2. The system ofclaim 1, wherein the first and the second spaces are divided intopatches in a unit of plane and a plane not more than a predeterminedminimum area is excluded from the patch and a plane not less than apredetermined maximum area is subdivided into multiple patches.
 3. Thesystem of claim 1, wherein the information on the geometricalcharacteristics of the patches includes at least some of sizes, heights,normal vectors, and aspect ratios thereof.
 4. The system of claim 1,wherein the corresponding location searching part provides the human Ywith information on the determined destination and offers user interfacecapable of receiving feedback from the human Y about whether thedetermined destination is appropriate.
 5. The system of claim 1, furthercomprising: a human gaze recognition part for acquiring information ongazes of the humans X and Y in the first and the second spaces.
 6. Thesystem of claim 1, wherein the space recognition part recognizes atleast one of types, locations, and orientations of objects existing inthe first and the second spaces.
 7. The system of claim 1, wherein, ifthe human X exists in a specified direction at a specified location ofthe first space and the human Y exists in a specific direction at aspecific location of the second space, the corresponding locationsearching part allows the avatar Y′ to be placed in the first space tohave a direction of 180 degrees (or near 180 degrees) from a directionof the human X and have a location within a predetermined distance fromthe human X and allows the avatar X′ to be placed in the second space tohave a direction of 180 degrees (or near 180 degrees) from a directionof the human Y and have a location within a preset distance from thehuman Y.
 8. The system of claim 7, wherein, if the first and the secondspaces are divided into multiple patches, the corresponding locationsearching part allows the avatar Y′ to be placed in a patch with a highdegree of appropriateness for a human to be located in the first spacewhile having the direction of 180 degrees (or near 180 degrees) from adirection of the human X and having a location within the predetermineddistance from the human X and allows the avatar X′ to be placed in apatch with a high degree of appropriateness for a human to be located inthe second space while having the direction of 180 degrees (or near 180degrees) from a direction of the human Y and having a location withinthe preset distance from the human Y.
 9. A method for providingtelepresence by employing avatars, comprising steps of: (a) acquiring,via a space recognition part including at least one sensor, informationregarding at least a first space, which is a present space, and a secondspace, which is a remote space; (b) acquiring, via a human motionrecognition part including at least one sensor, information on locationsof a human X in the first space and a human Y in the second space; (c)determining, via a corresponding location searching part, a location inthe first space where an avatar Y′, by referring to (i) information onthe first space and the second space, (ii) information on locations ofthe humans X and Y and an avatar X′ in the second space, and (iii)information on gazes of the humans X and Y in the first and secondspaces; wherein the avatar Y′ corresponds to the human Y, and the avatarX′ corresponds to the human X, wherein the corresponding locationsearching part estimates the destination in a unit of patch by dividingat least part of the first and the second spaces into patches, anddetermines the location where the avatar Y′ will be placed, wherein thecorresponding location searching part determines the location in thefirst space where the avatar Y′ will be placed by selecting a locationthat has the highest degree acquired by referring to (i) information ona degree of intrinsic similarity calculated by using at least some ofinformation on types of objects corresponding to the patches included inthe first and the second spaces and at least some of information ongeometrical characteristics of the patches; and (ii) information on adegree of spatial similarity calculated by using information on arelative location and a relative direction between the human X and theavatar Y′ in the first space and those between the human Y and theavatar X′ in the second space, wherein the corresponding locationsearching part determines a destination in the second space where thehuman Y intends to reach from the initial state by referring to at leastpart of (1) information on a function which increases as cumulativevalues of respective probabilities of gazing at the respective patchesduring a certain time section increase, (2) information on a functionfor evaluating degrees of appropriateness of locations for the human Yto be located by considering types of objects to which the respectivepatches correspond and geometrical characteristics of the respectivepatches, and (3) information on a function which increases asfrequencies of the respective patches being the destination actuallyreached by the human Y become higher; and (d) creating, via an avatarmotion creating part, a motion of the avatar Y′ by referring toinformation on the determined location where the avatar Y′ will beplaced; wherein the avatar motion creating part creates a motion of theavatar Y′ by searching a route to avoid interpenetration in the firstspace, and wherein the avatar motion creating part creates the motion ofthe avatar Y′ by referring to (i) a degree of difference between a poseof the human Y and that of the avatar Y′ and (ii) a degree ofpenetration of the avatar Y′ into an environment of the first space. 10.The method of claim 9, wherein, at the step of (c), if the human Xexists in a specified direction at a specified location of the firstspace and the human Y exists in a specific direction at a specificlocation of the second space, the avatar Y′ is allowed to be placed inthe first space to have a direction of 180 degrees (or near 180 degrees)from a direction of the human X and have a location within apredetermined distance from the human X and the avatar X′ is allowed tobe placed in the second space to have a direction of 180 degrees (ornear 180 degrees) from a direction of the human Y and have a locationwithin a preset distance from the human Y.
 11. One or morenon-transitory computer-readable recording media having stored thereon acomputer program that, when executed by one or more processors, causesthe one or more processors to perform acts including: a spacerecognition part including at least one sensor, said space recognitionpart acquiring information regarding at least a first space, which is apresent space, and a second space, which is a remote space; a humanmotion recognition part including at least one sensor, said human motionrecognition part acquiring information on locations of a human X in thefirst space and a human Y in the second space; (A) a correspondinglocation searching part for determining a location in the first spacewhere an avatar Y′, by referring to (i) information on the first spaceand the second space, (ii) information on locations of the humans X andY and an avatar X′ in the second space, and (iii) information on gazesof the humans X and Y in the first and second spaces; wherein the avatarY′ corresponds to the human Y, and the avatar X′ corresponds to thehuman X, wherein the corresponding location searching part estimates thedestination in a unit of patch by dividing at least part of the firstand the second spaces into patches, and determines the location wherethe avatar Y′ will be placed, wherein the corresponding locationsearching part determines the location in the first space where theavatar Y′ will be placed by selecting a location that has the highestdegree acquired by referring to (i) information on a degree of intrinsicsimilarity calculated by using at least some of information on types ofobjects corresponding to the patches included in the first and thesecond spaces and at least some of information on geometricalcharacteristics of the patches; and (ii) information on a degree ofspatial similarity calculated by using information on a relativelocation and a relative direction between the human X and the avatar Y′in the first space and those between the human Y and the avatar X′ inthe second space, wherein the corresponding location searching partdetermines a destination in the second space where the human Y intendsto reach from the initial state by referring to at least part of (1)information on a function which increases as cumulative values ofrespective probabilities of gazing at the respective patches during acertain time section increase, (2) information on a function forevaluating degrees of appropriateness of locations for the human Y to belocated by considering types of objects to which the respective patchescorrespond and geometrical characteristics of the respective patches,and (3) information on a function which increases as frequencies of therespective patches being the destination actually reached by the human Ybecome higher; and (B) an avatar motion creating part for creating amotion of the avatar Y′ by referring to information on the determinedlocation where the avatar Y′ will be placed; wherein the avatar motioncreating part creates a motion of the avatar Y′ by searching a route toavoid interpenetration in the first space, and wherein the avatar motioncreating part creates the motion of the avatar Y′ by referring to (i) adegree of difference between a pose of the human Y and that of theavatar Y′ and (ii) a degree of penetration of the avatar Y′ into anenvironment of the first space.